Skip to content

feat(providers): add profile-backed policy composition#1037

Merged
johntmyers merged 15 commits intomainfrom
feat/947-provider-profile-composition-draft
May 5, 2026
Merged

feat(providers): add profile-backed policy composition#1037
johntmyers merged 15 commits intomainfrom
feat/947-provider-profile-composition-draft

Conversation

@johntmyers
Copy link
Copy Markdown
Collaborator

@johntmyers johntmyers commented Apr 29, 2026 •
edited
Loading

Summary

Add the first provider profile foundation: bundled YAML provider profiles, profile proto surfaces, CLI discovery for provider types, and JIT policy composition behind the opt-in use_providers_v2 gateway setting.

Related Issue

Closes #947

UX Changes

  • Adds openshell provider list-types to browse built-in provider profiles exposed by the gateway.
  • Provider profile categories are now a proto enum: other, inference, agent, source_control, messaging, data, and knowledge.
  • Default behavior is unchanged: with use_providers_v2=false or unset, providers keep the existing credential discovery/injection behavior and profile-backed policy composition is not used.
  • When use_providers_v2=true is enabled at the gateway, sandbox policy reads compose the sandbox-authored policy with provider profile policy layers JIT. Built-in profile YAML defines the provider policy defaults, and duplicate/overlapping user policy entries are preserved through layered composition instead of replacing the user's sandbox policy.
  • Gateway-global policy remains a full override in both modes. It acts as the emergency switch: operators can rapidly replace effective sandbox policy fleet-wide without provider profile layers adding extra egress.
  • Credential injection is unchanged in both modes for this PR. Provider credentials still resolve through the existing legacy path until a future iteration starts materializing credentials from provider profiles.

Changes

  • Add provider profile protos, category enum, and bundled per-provider YAML profiles.
  • Add profile registry support and CLI provider list-types discovery.
  • Add opt-in use_providers_v2 gateway setting for profile-backed policy composition.
  • Compose provider profile policy layers JIT when the gateway setting is enabled and the effective policy source is sandbox-scoped.
  • Preserve global policy as a full emergency override that suppresses provider profile layers.
  • Keep generic as a legacy provider type without a bundled v2 profile or automatic policy contribution.
  • Add policy composition tests and provider profile parsing/catalog coverage.
  • Update architecture and provider docs for the new profile model.

Testing

  • RUSTC_WRAPPER= cargo check -p openshell-core -p openshell-providers -p openshell-cli -p openshell-server
  • RUSTC_WRAPPER= cargo test -p openshell-providers
  • RUSTC_WRAPPER= cargo test -p openshell-cli cli_provider_types_match_registry
  • RUSTC_WRAPPER= cargo test -p openshell-server sandbox_config_
  • RUSTC_WRAPPER= cargo test -p openshell-server provider_environment_resolution_is_unchanged_by_providers_v2_setting
  • RUSTC_WRAPPER= cargo test -p openshell-server provider_profile
  • RUSTC_WRAPPER= cargo test -p openshell-cli provider_list_types_cli_uses_profile_browsing_rpc
  • mise run pre-commit passes
  • Unit tests added/updated
  • E2E tests added/updated (if applicable)

Checklist

  • Follows Conventional Commits
  • Commits are signed off (DCO)
  • Architecture docs updated (if applicable)

@copy-pr-bot
Copy link
Copy Markdown

copy-pr-bot Bot commented Apr 29, 2026

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@github-actions
Copy link
Copy Markdown

github-actions Bot commented Apr 29, 2026

🌿 Preview your docs: https://nvidia-preview-pr-1037.docs.buildwithfern.com/openshell

@johntmyers johntmyers force-pushed the feat/947-provider-profile-composition-draft branch from 89e8c63 to 68e6947 Compare April 30, 2026 16:27
@johntmyers johntmyers added the test:e2e Requires end-to-end coverage label Apr 30, 2026
@github-actions
Copy link
Copy Markdown

github-actions Bot commented Apr 30, 2026

Label test:e2e applied, but pull-request/1037 is at {"messa while the PR head is 68e6947. A maintainer needs to comment /ok to test 68e6947fd55b872dd43a621bc7c64059901a0cbb to refresh the mirror. Once the mirror catches up, re-run Branch E2E Checks from the Actions tab.

@johntmyers
Copy link
Copy Markdown
Collaborator Author

johntmyers commented Apr 30, 2026

/ok to test 68e6947

@johntmyers johntmyers marked this pull request as ready for review April 30, 2026 16:41
@johntmyers johntmyers requested a review from a team as a code owner April 30, 2026 16:41
TaylorMutch
TaylorMutch reviewed May 4, 2026
View reviewed changes
Comment thread providers/claude.yaml
TaylorMutch
TaylorMutch previously approved these changes May 4, 2026
View reviewed changes
drew
drew reviewed May 4, 2026
View reviewed changes
Comment thread crates/openshell-cli/src/run.rs
Comment thread crates/openshell-providers/src/profiles.rs
Comment thread architecture/gateway-settings.md Outdated
Comment thread architecture/sandbox-providers.md Outdated
Comment thread crates/openshell-providers/src/profiles.rs
@johntmyers johntmyers force-pushed the feat/947-provider-profile-composition-draft branch from 68e6947 to 1255bb9 Compare May 4, 2026 20:25
@johntmyers johntmyers force-pushed the feat/947-provider-profile-composition-draft branch from 1255bb9 to 62bfdcf Compare May 4, 2026 20:48
@johntmyers
Copy link
Copy Markdown
Collaborator Author

johntmyers commented May 4, 2026

/ok to test 62bfdcf

drew
drew previously approved these changes May 4, 2026
View reviewed changes
@johntmyers
Copy link
Copy Markdown
Collaborator Author

johntmyers commented May 5, 2026

/ok to test 8be0be6

@johntmyers
Copy link
Copy Markdown
Collaborator Author

johntmyers commented May 5, 2026

/ok to test a4af3b8

@johntmyers
Copy link
Copy Markdown
Collaborator Author

johntmyers commented May 5, 2026

/ok to test 4a0d593

@johntmyers
Copy link
Copy Markdown
Collaborator Author

johntmyers commented May 5, 2026

/ok to test 7b1d6ea

@johntmyers johntmyers merged commit 043bde2 into main May 5, 2026
64 of 66 checks passed
@johntmyers johntmyers deleted the feat/947-provider-profile-composition-draft branch May 5, 2026 01:34
This was referenced May 5, 2026
Open
Merged
TaylorMutch added a commit that referenced this pull request May 5, 2026
@TaylorMutch
fix(docker): copy providers/ into in-Docker rust-builder stage
d138767 
PR #1037 added include_str!("../../../providers/*.yaml") in
crates/openshell-providers/src/profiles.rs, but the BUILD_FROM_SOURCE=1
path of Dockerfile.images only COPY's Cargo.toml/Cargo.lock, crates/,
and proto/. With providers/ missing the cargo build inside the rust-
builder stage fails to read the embedded YAML. The release path is
unaffected because it copies pre-built binaries from
deploy/docker/.build/prebuilt-binaries/.

This breaks 'mise run e2e:helm:*' and any other workflow that builds
images from source via this Dockerfile (e.g., the local helm-e2e
harness). Add 'COPY providers/ providers/' alongside the other source
inputs.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@drew drew drew left review comments

@TaylorMutch TaylorMutch TaylorMutch left review comments

Assignees

No one assigned

Labels

test:e2e Requires end-to-end coverage

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

feat: add provider profile registry and policy layer composition foundation

3 participants

@johntmyers @drew @TaylorMutch